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Abstract 

This  paper  describes  the  systems  engineering  intervention  the  researchers 
implemented  in  their  study  of  patient  safety  in  outpatient  surgery.  The 
intervention  process  is  based  on  the  SEIPS  (Systems  Engineering  Initiative  for 
Patient  Safety)  model  of  work-system  and  patient  safety.  The  paper  provides 
details  on  the  steps  in  the  intervention  process  (e.g.,  overall  structure, 
decisionmaking  criteria  for  selection  of  intervention,  participants)  and  the  various 
data  collection  tools  and  methods  that  were  used  at  each  step  of  the  intervention 
process.  The  systems  engineering  intervention  process  consists  of  three  steps:  (1) 
defining  and  designing  the  content  and  the  implementation  plan  of  the 
intervention,  (2)  implementing  the  intervention,  and  (3)  institutionalizing  the 
intervention.  Data  collection  methods  used  for  defining  and  designing  the 
intervention  include  an  initial  employee  questionnaire  and  patient  shadowing.  An 
employee  questionnaire  and  a  patient  survey  are  the  two  methods  used  to  evaluate 
the  impact  of  the  systems  engineering  intervention. 


Introduction 

•  ••••••  ♦ 

In  our  Systems  Engineering  Initiative  in  Patient  Safety  (SEIPS)  study, 

“Patient  Safety  in  Outpatient  Surgery,”  we  have  developed  an  intervention 
process  that  targets  changes  in  health  care  systems  with  the  aim  of  improving  both 
patient  safety  and  employee  and  organizational  outcomes.1  In  this  paper,  we 
describe  our  systems  engineering  intervention  process.  Our  SEIPS  study  in 
outpatient  surgery  is  used  as  example  of  the  application  of  the  intervention 
process. 

There  is  an  underlying  conceptual  framework  that  drives  the  systems 
engineering  intervention  process.  In  the  SEIPS  model  of  work  system  and  patient 
safety  (Figure  l),1  we  integrate  Donabedian’s  Structure-Process-Outcome 
framework2  and  the  work  system  model  developed  by  Smith  and  Carayon. '' 4  The 
structure  of  an  organization  (or  more  generally,  the  work  system)  affects  the 
extent  to  which  safe  care  is  provided  (the  process).  The  means  of  caring  for  and 
managing  the  patient  (the  process)  affects  the  likelihood  the  patient  completes  his 
or  her  experience  without  harm  (outcome).  It  also  influences  employee  and 


For  additional  information  on  SEIPS,  please  consult  the  SEIPS  Web  site  at 
http  ://www2.  fpm.  wise  .edu/  seips/. 
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Figure  1.  Systems  Engineering  Initiative  in  Patient  Safety  (SEIPS)  model  of  work  system 
and  patient  safety 


WORK  SYSTEM  PROCESS  OUTCOMES 


organizational  outcomes. 1  The  goal  of  the  systems  engineering  intervention  is  to 
make  changes  to  the  structure  and  process  elements  to  improve  patient  outcomes 
as  well  as  to  improve  employee  and  organizational  outcomes. 

This  paper  provides  details  on  the  steps  in  the  intervention  process  (e.g., 
overall  structure,  decisionmaking  criteria  for  selection  of  an  intervention, 
participants)  and  the  various  data  collection  tools  and  methods  that  can  be  used  at 
each  step  of  the  intervention  process.  We  also  emphasize  the  major  challenges  in 
implementing  systems  engineering  interventions  aimed  at  improving  patient 
safety  and  strategies  that  can  be  used  to  mitigate  those  challenges.  The  specifics 
of  the  intervention  (the  content  and  implementation  plan  of  the  intervention)  need 
to  be  adapted  to  the  particular  organization.  However,  the  intervention  process 
itself  can  be  replicated  in  any  organization. 


The  organizational  context  for  intervention 

In  this  paper,  we  describe  the  systems  engineering  intervention  process  as  it 
was  applied  to  our  SEIPS  pilot  project.  The  study  was  conducted  in  collaboration 
with  five  outpatient  surgery  centers  located  in  Madison,  Wisconsin.  One  surgery 
center  is  hospital-based  and  is  managed  by  a  physician-directed  corporate  entity. 
Excluding  medical  staff,  it  employs  39  full-time  equivalent  employees  (FTEs). 
Another  surgery  center  is  free-standing  and  is  managed  by  the  same  physician- 
directed  corporate  entity.  Excluding  medical  staff,  it  employs  22.1  FTEs.  Two 
other  centers  are  managed  under  the  auspices  of  a  joint  venture  between  a  large 
medical  group  and  the  hospital  at  which  the  group’s  physicians  primarily  practice. 
Excluding  medical  staff,  both  surgery  centers  employ  49  FTEs.  The  fifth  surgery 
center  is  a  government-run  provider  of  ambulatory  and  inpatient  care,  and 
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employs  18.2  FTEs,  excluding  medical  staff.  The  total  patient  volume  of  the 
combined  surgery  centers  was  about  25,000  cases  in  fiscal  year  2001-2002. 

All  five  outpatient  surgery  centers  are  members  of  the  Madison  Patient  Safety 
Collaborative  (MPSC).  MPSC  was  established  in  September  2000  when 
Madison’s  hospitals  and  medical  groups  collectively  decided  to  form  a  group 
focused  on  developing  and  implementing  patient  safety  practices  to  further 
improve  the  safety  of  care  received  by  all  consumers  in  the  greater  Madison  area 
(additional  infonnation  is  available  at  www.madisonpatientsafety.org).  The 
collaborative  was  founded  on  the  tenets  that  patient  safety  is  a  common  goal  of 
health  care  institutions  rather  than  a  competitive  issue,  and  that,  through 
cooperation,  Madison  providers  will  be  able  to  implement  safety  improvements 
that  go  well  beyond  the  scope  of  any  single  organizational  effort. 

Each  member  of  the  collaborative  has  made  an  organizational  commitment  to 
improving  patient  safety  by  dedicating  infrastructure,  intellectual  capital,  and 
fiscal  resources  to  MPSC  projects.  With  funding  from  five  members,  the  MPSC 
has  established  a  community-based  center  to  coordinate,  record,  and  communicate 
patient  safety  initiatives.  The  center  has  identified  leaders  to  coordinate  efforts 
among  member  organizations,  share  “lessons  learned”  and  progress  achieved, 
secure  external  support  for  MPSC  activities,  and  establish  key  partnerships  with 
local  and  national  organizations  concerned  with  patient  safety.  The  SEIPS  project 
on  patient  safety  in  outpatient  surgery  was  conducted  under  the  auspices  of 
MPSC. 

The  collaborative  intervention  process 

Data  obtained  during  the  initial  data  collection  phase  were  analyzed  (see 
section  below  on  “Initial  data  collection”)  and  presented  to  the  outpatient  surgery 
centers.  The  sites  collectively  decided  to  make  changes  to  the  pre-operative 
process  with  the  goal  of  improving  the  coordination  and  communication  of  care 
among  providers.  In  addition,  the  current  patient  follow-up  process  at  each  site 
was  to  be  changed  to  become  more  active.  Four  meetings  were  held  with 
representatives  of  the  outpatient  surgery  centers  and  the  research  teams  to  discuss 
the  initial  data,  identify  areas  of  opportunity  for  improvement,  and  decide  the 
nature  of  the  intervention.  The  content  of  each  of  these  four  meetings,  as  well  as 
the  rest  of  the  meetings,  is  described  in  Table  1.  The  first  two  meetings  were 
primarily  led  by  the  research  team.  At  “meeting  #3”  and  at  subsequent  meetings, 
representatives  of  the  five  surgery  centers  contributed  to  most  of  the  interaction 
during  the  meetings.  Starting  with  “meeting  #  5,”  each  of  the  surgery  centers  was 
asked  to  provide  an  update  on  their  activities,  and  they  became  even  more 
engaged  in  the  collaborative  intervention  process. 

It  is  important  to  note  that  at  each  of  the  meetings,  each  of  the  five  surgery 
centers  was  represented  by  at  least  one  individual  (and  in  most  cases  two 
individuals).  The  individuals  participating  in  the  meetings  included  the  physician- 
directors  of  two  surgery  centers,  the  vice-president  and  manager  of  two  surgery 
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Table  1.  Collaborative  intervention  process 


Meeting 

Date 

Attendance 

Content 

Meeting  #1 

October  2002 

•  7  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  7  representatives  from  the  5 
outpatient  surgery  centers 

•  Presentation  of  SEIPS 
project 

•  Discussion  of  project 
objectives  and  timeline 

•  Discussion  of  initial  data 
collection 

•  Brainstorming  discussion  on 
patient  safety  issues 
experienced  by  the  surgery 
centers 

•  Organization  of  project 

Meeting  #2 

February  2003 

•  3  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  7  representatives  from  the  5 
outpatient  surgery  centers 

•  Presentation  of  initial  data 
collection  results 

•  Discussion  of  criteria  for 
selecting  intervention 

•  Discussion  of  timeline 

Meeting  #3 

March  2003 

•  4  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  5  representatives  from  the  5 
outpatient  surgery  centers 

•  Discussion  of  initial  data 
analysis 

•  Brainstorming  discussion  of 
possible  interventions 

Meeting  #4 

March  2003 

•  7  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  6  representatives  from  the  5 
outpatient  surgery  centers 

•  Agreement  on  content  of 
intervention 

•  Discussion  of  main  data 
collection  tools  and 
procedures 

Meeting  #5 

July  2003 

•  5  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  9  representatives  from  the  5 
outpatient  surgery  centers 

•  Presentation  of  preliminary 
employee  questionnaire 
data  analysis 

•  Update  on  intervention  by 
each  of  the  surgery  centers 

Meeting  #6 

February  2004 

•  6  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  1 1  representatives  from  the 

5  outpatient  surgery  centers 

•  Presentation  of  employee 
questionnaire  data  and 
patient  survey  data 

•  Update  on  intervention  by 
each  of  the  surgery  centers 

•  Discussion  of  post¬ 
intervention  data  collection 

Meeting  #7 

September  2004 

•  3  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  6  representatives  from  the  5 
outpatient  surgery  centers 

•  Update  on  post-intervention 
data  collection 

•  Update  on  intervention  by 
each  of  the  surgery  centers 

Meeting  #8 

November  2004 

•  5  SEIPS  research  team 
members 

•  MPSC  coordinator 

•  4  representatives  from  the  5 
outpatient  surgery  centers 

•  Presentation  of  final  data 
analysis 

•  Discussion  of  lessons 
learned  and 
institutionalization 

•  Presentation  of  followup 
project  on  pre-operative 
information  flow  at  one 
surgery  center 
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centers,  the  nurse  managers  of  all  five  surgery  centers,  and  patient  safety  and 
quality  improvement  personnel  of  each  of  the  surgery  centers. 

Steps  in  the  intervention  process 

Designing  or  redesigning  work  systems  and  processes  can  represent  a  major 
organizational  investment,  requiring  the  involvement  of  numerous  people; 
substantial  time  to  conduct  evaluations,  analyze  data,  and  design  and  implement 
solutions;  adequate  resources;  and  sufficient  expertise  and  knowledge.  Like  any 
major  organizational  or  technological  change,  system  redesign  needs  to  be 
managed.  A  process  needs  to  be  implemented  for  coordinating  all  the  personnel, 
activities,  and  resources  involved  in  the  redesign  project.  Many  work  design 
processes  have  been  identified  and  proposed.5  They  take  different  forms,  have 
different  levels  of  specifications,  and  use  different  tenninologies.  Wilson5 
proposes  an  ergonomics  design  process  with  12  steps: 

1 .  Setting  objectives  and  requirements  for  analysis. 

2.  Initiating  consultation  and  participation. 

3.  Identifying  priorities  and  critical  issues  for  data  collection. 

4.  Conducting  task  analysis  and  synthesis. 

5.  Conducting  task  design,  allocation,  and  division  of  function. 

6.  Analyzing  workspace  layout. 

7.  Analyzing  work  environment. 

8 .  Analyzing  j  ob  design. 

9.  Analyzing  job  aids,  manuals,  procedures,  and  training  specifications. 

10.  Analyzing  work  organization. 

1 1 .  Evaluating  and  redesigning  the  aspects  of  work. 

12.  Monitoring  the  redesign. 

This  ergonomics  design  process  is  very  detailed  with  regard  to  analysis,  but 
may  lack  details  and  specifications  for  redesign  implementation  and  evaluation. 
The  structured  work  redesign  process  proposed  by  Parker  and  Wall6  includes 
eight  phases: 

1 .  Set  the  direction. 

2.  Diagnose  the  situation. 

3 .  Formulate  the  work  design. 

4.  Consider  the  wider  context. 

5.  Plan  the  implementation. 

6.  Conduct  a  prechange  assessment. 

7.  Implement,  re-assess,  evaluate,  fine-tune. 
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8.  Spread  the  work  design  and  sustain  the  change. 

We  have  synthesized  Wilson’s  12-step  process5  and  Parker  and  Wall’s  8-step 
process6  into  a  3-step  intervention  process  as  follows: 

1.  Definition  and  design  of  the  intervention: 

a.  determine  the  content  of  the  intervention 

b.  determine  the  implementation  plan  of  the  intervention 

2.  Implementation  of  the  intervention 

3.  Institutionalization  of  the  intervention. 

Steps  1-10  of  Wilson  and  steps  1-6  of  Parker  and  Wall  are  included  in  our 
first  step  of  defining  and  designing  the  intervention.  Step  5  of  Parker  and  Wilson 
is  similar  to  our  second  substep  on  determining  the  implementation  plan.  Step  1 1 
of  Wilson  and  step  7  of  Parker  and  Wall  are  similar  to  our  second  step.  Step  12  of 
Wilson  and  step  8  of  Parker  and  Wall  are  similar  to  our  third  step. 

Defining  and  designing  the  intervention 

In  the  first  step  of  the  systems  engineering  intervention  process,  the  specific 
problem  to  be  solved  needs  to  be  identified  (i.e.,  defining  the  content  of  the 
intervention).  This  step  involves  steps  1-10  of  Wilson’s  ergonomics  design 
process.5  Following  Wilson’s  recommendation  and  based  on  the  work  system 
model  of  Smith  and  Carayon,3,4  we  examined  a  range  of  system  variables  (e.g., 
tasks,  tools  and  technologies,  workspace  layout,  work  environment,  job  design, 
work  organization).  The  methods  used  to  conduct  this  analysis  are  described 
below  (see  the  section  on  “Initial  data  collection”).  After  a  phase  of  initial  data 
collection  and  analysis,  an  intervention  is  decided  upon  (phase  3  of  Parker  and 
Wall’s  work-redesign  process6).  Several  elements  come  into  play  into  this 
decisionmaking  process.  The  team  agreed  that  certain  criteria  were  to  be  met 
when  selecting  the  cross-center  intervention.  These  criteria  included: 

•  the  intervention  coincides  with  SEIPS  objectives 

•  all  organizations  are  willing  to  participate 

•  all  organizations  are  committed — need  to  identify  a  contact  person 

•  the  intervention  provides  balance  between  research  and  practice  and 
provides  high  research  rigor 

•  the  intervention  is  clinically  relevant 

•  the  timeline  is  feasible 

•  data  evaluation  can  occur  (measurement  sensitivity) 

•  the  magnitude  of  impact  on  patient  safety  is  substantial 

•  the  intervention  is  sustainable 

•  the  intervention  fits  the  budget  and  resources  available 
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Given  the  theoretical  basis  of  the  SEIPS  model  and  the  systems  engineering 
intervention,  we  attempted  to  identify  interventions  that  affect  both  the  work 
system  and  patient  safety.  Figure  2 A  shows  the  kind  of  intervention  that  we  were 
looking  for.  The  intervention  was  to  have  a  high  impact  on  both  the  work  system 
and  patient  safety. 

The  goal  of  the  systems  engineering  intervention  is  to  affect  patient  outcomes 
as  well  as  both  individual  and  organizational  outcomes.  The  patient  safety  issues 
of  interest  in  our  SEIPS  study  conducted  in  outpatient  surgery  are  as  follows: 

•  providers  are  fully  informed  of  their  patient’s  clinical  status 

•  the  surgery  is  perfonned  appropriately  (e.g.,  correct  site  surgery)  and 
clinically  accurate 

•  patients  are  adequately  prepared  pre-operatively 

•  patients  are  well  educated  for  post-operative  self-management 
(including  an  understanding  of  their  medications) 

In  determining  the  content  of  the  intervention,  the  SEIPS  team  targeted 
specific  patients  for  the  intervention  and  the  evaluation  (Figure  2B).  The  targeted 
patients  involve  “high  acuity”  patients  as  well  as  those  undergoing  high-risk 
procedures.  We  chose  this  strategy  for  a  number  of  reasons.  First,  we  recognized 
that  having  a  fully  representative  sample  of  patients  from  each  surgery  center 
would  include  a  proportion  of  patients  and  procedures  that  present  minimal  risk  or 
that  would  be  unique  to  the  respective  center.  We  would,  for  that  reason,  have 
needed  a  larger  sample  of  patients  to  obtain  any  type  of  statistically  meaningful 
results.  Therefore,  by  choosing  “at-risk”  patients  as  our  subjects  we  were  more 
likely  to  obtain  greater  variation  in  responses.  Secondly,  we  had  limited  time  and 
resources  to  conduct  the  intended  study.  Finally,  we  had  to  identify  homogenous 
patient  groups  among  the  five  centers.  The  types  of  procedures  selected  allowed 
us  to  then  perform  cross-center  comparative  analyses. 


Figure  2A.  Determining  a  systems  engineering  intervention 

Impact  on  work  system 


Impact  on 

patient  safety 
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Figure  2B.  Targeting  patients  for  the  intervention  and  the  evaluation 

Risk  -  patient 


LOW 


HIGH 


Implementation  of  the  intervention 

Using  the  process  described  above,  along  with  information  captured  from  the 
initial  data  collection,  the  team  chose  to  address  two  themes  in  the  intervention: 

(1)  the  adequacy  of  patients’  pre-operative  clinical  infonnation,  and  (2)  active 
patient  follow-up.  At  this  point,  each  outpatient  surgery  center  had  to  detennine 
strategies  it  would  employ  to  attempt  to  ensure  that  it  had  all  of  the  clinically 
relevant  and  necessary  information  needed  to  safely,  accurately,  and  appropriately 
proceed  with  surgery.  A  variety  of  “interventions”  resulted,  including  (1) 
implementing  an  institutional  policy  requiring  that  all  patient  information  is 
provided  to  the  surgery  center  by  noon  on  the  working  day  prior  to  the  surgery; 

(2)  updating  center  policy-and-procedure  manuals  and  then  distributing  them  to 
the  offices  of  relevant  referring  physicians;  (3)  integrating  relevant  clinical 
information  in  the  medical  center’s  newly  implemented  electronic  medical  record, 
and  providing  the  surgery  centers  with  access  to  this  infonnation;  and  (4)  creating 
an  electronic  anesthesia  clinic  record  that  logs  necessary  clinical  information  pre- 
operatively. 

Each  outpatient  surgery  center  developed  its  own  specific  implementation 
plan.  What  each  site  chose  to  change,  as  well  as  how  they  implemented  the 
change,  depended  on  the  unique  systems  in  place  at  each  site.  In  addition,  the 
research  team  met  with  each  of  the  surgery  centers  separately  to  refine  the 
implementation  plan  as  well  as  the  specifics  of  the  intervention  to  be  implemented 
at  each  site.  These  meetings  were  guided  by  findings  of  the  initial  data  collection 
(i.e.,  the  intervention  should  address  issues  identified  in  the  initial  effort),  the  two 
themes  (pre-operative  patient  assessment  and  active  patient  follow-up)  that  were 
collectively  identified  for  the  intervention,  the  process  analysis  performed  for 
each  of  the  surgery  centers,  center-specific  criteria  for  selection/design  of  the 
intervention,  and  center-specific  patient  safety  issues. 
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Data  collection 

In  the  systems  engineering  intervention  process,  data  are  collected  and  used  to 
design  the  intervention  and  to  evaluate  the  impact  of  the  intervention  as  follows: 

•  Data  to  define  and  design  the  intervention.  This  included  various 
tools  and  data  sources — an  initial  questionnaire  to  gather  staff  input, 
patient  shadowing  to  evaluate  the  patient  care  process,  flowcharting 
and  process  analysis,  review  of  various  data  sources  (e.g.,  physical 
layout,  data  on  quality  and  safety  of  care),  and  a  participatory  design 
process. 

•  Data  to  evaluate  the  impact  of  the  intervention  at  various  levels. 

Data  were  collected  pre-  and  post-intervention  to  evaluate  the  impact 
of  the  intervention  on  staff  and  patients.  These  data  were  collected  by 
means  of  an  employee  questionnaire  and  a  patient  survey  that 
addressed  the  different  elements  of  the  SEIPS  model  of  work  system 
and  patient  safety. 

Initial  data  collection 

To  begin  our  study,  we  needed  to  gain  an  overall  understanding  of  the  various 
participating  surgery  centers.  We  accomplished  this  by  collecting  position 
descriptions  for  all  of  the  health  care  provider  roles  at  each  site;  obtaining  floor 
plans  for  each  surgery  center;  asking  staff  to  complete  surveys  aimed  at 
identifying  both  the  quality  of  care  and  the  patient  safety  issues  they  perceive 
exist  at  their  center  (as  well  as  the  performance  obstacles  they  encounter  in  their 
work);  and  “shadowing”  patients  (having  a  member  of  the  research  staff 
accompany  the  patient  through  the  entire  experience  at  the  center)  to  capture  the 
outpatient  surgery  experience  from  the  patient’s  perspective.  Our  interaction  with 
the  individual  surgery  centers  also  involved  flowcharting  and  process  analysis  of 
the  outpatient  surgical  experience.7  The  participatory  process  analysis  involved 
gathering  input  from  the  surgery  centers  regarding  steps  of  the  outpatient  surgery 
process,  flowcharting  the  process,  discussing  the  process  with  the  center  staff,  and 
making  changes  to  the  flowchart.  In  this  paper,  we  briefly  describe  two  of  the 
initial  data  collection  methods:  the  initial  employee  questionnaire  and  patient 
shadowing. 

Initial  employee  questionnaire 

The  objective  of  the  initial  employee  questionnaire  was  to  evaluate  employee 
perceptions  of  quality-  and  safety-of-care  issues,  and  to  assess  the  factors  in  the 
work  environment  that  facilitate  or  hinder  employee  perfonnance  (i.e., 
performance  facilitators  and  obstacles).  While  attending  staff  meetings  at  each  of 
the  sites,  we  introduced  and  explained  our  research  project  and  then  distributed 
questionnaires  to  all  staff  present.  Attendance  was  taken  at  each  site’s  single 
meeting,  and  questionnaires  were  later  distributed  by  supervisory  staff  to  those 
not  present  at  the  meeting.  The  presentations  were  well  received  and  staff 
expressed  interest  and  a  willingness  to  participate  in  the  study.  We  attached 
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Institutional  Review  Board-required  cover  sheets  to  all  of  the  questionnaires  (in 
lieu  of  obtaining  subjects’  individual  written  consent)  to  ensure  that  staff  fully 
understood  the  intent  and  consequences  of  participation.  We  also  offered  staff  the 
option  of  completing  paper  or  electronic  versions  of  the  questionnaire.  The 
electronic  versions  included  a  Web-based  format  and  an  RTF-formatted  file  on  a 
diskette.  Seventy-seven  of  the  79  responses  received  were  completed  on  the  paper 
version,  and  two  respondents  utilized  the  Web  format.  Completed  paper  surveys 
were  deposited  in  locked  mailboxes  at  each  site  to  assure  confidentiality. 

Research  staff  collected  the  surveys  twice  a  week  for  the  3  weeks  during  which 
the  surveys  could  be  completed  and  returned. 

The  initial  employee  questionnaire  included  the  following  three  items  plus 
specific  context  and  directions  for  each: 

1 .  What  do  you  think  are  the  main  quality  of  care  and  patient  safety 
issues  in  your  outpatient  surgery  center? 

2.  Please  think  of  instances  in  the  past  year  when  you  feel  your 
performance  was  challenged  or  below  par  due  to  problems  in  the 
outpatient  surgery  center  “system.” 

3.  Please  think  of  instances  in  the  past  year  when  you  were  able  to 
perform  your  job  very  well. 

We  then  performed  content  analysis  on  responses  to  all  three  of  the  survey 
questions.  The  first  question  asked  staff  to  identify  quality-of-care  and  patient 
safety  issues  they  believe  affect  patient  outcomes  and  experience  at  specific 
stages  of  the  outpatient  surgery  process — from  the  work-up  (prior  to  presenting 
for  surgery)  to  patient  follow-up  at  home.  The  second  question  asked  staff  to 
identify  “instances  when  your  perfonnance  was  challenged  or  below  par,”  and 
question  three  asked  staff  to  describe  “instances  when  you  were  able  to  perform 
your  job  very  well.”  The  format  of  the  questionnaire  included  one  blank  page  per 
question.  The  questions  were  written  and  clarified  by  providing  examples  listed  at 
the  top  of  the  page.  This  was  done  so  that  staff  might  understand  the  number  and 
magnitude  of  issues  they  could  consider  and  include  in  their  responses.  Figure  3 
shows  data  that  were  presented  to  the  five  surgery  centers  and  summarizes  initial 
employee  questionnaire  data. 

The  overall  performance-obstacles  approach  used  in  the  initial  employee 
questionnaire  is  described  in  the  paper  on  “Performance  Obstacles  and  Facilitators 
of  Health  Care  Providers”  to  be  published  in  Volume  4  of  the  Organizational 

o 

Psychology  and  Health  Care  book  series.  Data  obtained  during  the  initial  data 
collection  of  our  outpatient  surgery  study  are  reported  in  a  paper  that  was 
presented  at  the  2003  Conference  of  the  Human  Factors  and  Ergonomics  Society9 
and  in  a  paper  submitted  for  publication.10  These  publications  include  details 
about  data  collection,  data  analysis,  and  results. 

Patient  shadowing 

We  also  performed  “shadowings”  on  patients  undergoing  outpatient  surgery  to 
better  understand  the  complexity  of  the  outpatient  surgery  experience  from  the 
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Figure  3.  Analysis  of  initial  employee  questionnaire  data 


Staff  Responses  of  Instances  When  Able  to  Perform  Job 
Very  Well 
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Preliminary  results  of  the  staff  completion  of  an  open-ended  questionnaire  aimed  at 
identifying  the  areas  of  concern  staff  have  regarding  quality  and  patient  safety,  as  well  as  those 
aspects  of  the  work  units  and  their  associated  systems  that  promote  patient  safety  and  a  healthy 
work  environment. 


patient’s  perspective.  The  work  system  model  served  as  the  basis  for  summarizing 
the  experience.3'4  Individuals  with  clinical  training  and  expertise  followed 
patients  having  outpatient  surgery  from  the  point  of  admission  to  the  unit/center 
until  discharge.  There  was  no  intent  to  scrutinize  the  quality  of  care  provided  by 
the  providers  (e.g.,  the  surgical  technique). 

Patients  were  chosen  by  the  medical  director  of  each  unit  based  primarily  on 
the  complexity  of  the  case  and  the  availability  of  the  individual  to  shadow  the 
patient.  The  patient’s  surgeon  also  had  to  agree  to  the  shadowing.  To  eliminate 
any  difficulties  in  communication,  only  individuals  who  spoke  English  fluently 
were  considered.  Once  the  patient  agreed  with  his/her  surgeon  to  be  “shadowed” 
(after  being  given  an  overview  of  the  shadowing  process  that  included  following 
the  patient  and  recording  observations  on  a  data  collection  form),  written  consent 
was  obtained  from  the  patient  by  the  “shadower.”  The  patient  was  assured  that 
participation  was  totally  voluntary  and  that  no  identifiable  information  concerning 
them  or  those  participating  in  their  care  would  be  collected.  Minimal  verbal 
interaction  occurred  between  the  researcher  and  patient,  and  only  questions  such 
as:  “How  far  did  you  travel  to  get  here?”  and  “How  long  ago  was  your  surgery 
scheduled?”  were  asked.  All  responses  were  ultimately  summarized  and  reported 
using  a  work  system  worksheet.  Using  this  procedure,  a  total  of  12  patient 
shadowings  were  performed  at  five  different  outpatient  surgery  centers.  This 
patient  shadowing  method  is  further  described  in  a  paper  that  was  presented  at  the 
2003  Congress  of  the  International  Ergonomics  Association.11  Table  2  displays  an 
example  of  data  collected  in  the  patient  shadowing. 
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Main  data  collection 

In  order  to  evaluate  the  systems  engineering  intervention,  we  utilized  two 
means  of  data  collection:  patient  telephone  interviews  and  an  employee 
survey/questionnaire.  Patient  surveys  and  questionnaire  data  are  collected  at  two 
points  in  time:  immediately  prior  to  and  12  months  after  the  implementation  of 
the  intervention. 

Employee  questionnaire 

All  employees  working  in  the  five  outpatient  surgery  centers  were  asked  to 
voluntarily  respond  to  an  employee  survey.  A  cover  letter  and  infonnation  sheet 
explaining  the  study  accompany  each  questionnaire.  An  employee’s  completion 
of  the  written  or  electronic  version  of  the  survey  implied  their  consent  to 
participate.  Because  of  the  need  to  track  the  results  by  respondent  over  time, 
employees  were  asked  to  record  a  unique,  easily  remembered  number  to  allow  for 
tracking  and  analysis  (the  final  four  digits  of  their  social  security  number).  The 
surveys  were  distributed  to  all  staff  at  the  beginning  of  their  daily  rounds  when  an 
overview  of  the  survey  was  presented.  The  surveys  were  returned  via  a  locked 
secure  drop  box  at  any  of  the  five  sites,  U.S.  or  University  interdepartmental  mail, 
or  via  the  Internet  if  they  chose  to  complete  the  web  version  of  the  survey. 
Follow-up  written  and  verbal  reminders  were  used  to  encourage  participation  by 
the  staff. 

The  objective  of  the  employee  questionnaire  was  to  evaluate  staff  perceptions 
of  (1)  work  system  elements  targeted  or  possibly  affected  by  the  intervention 
(communication,  coordination,  workplace  facilities,  equipment  and  supplies, 
workload),  (2)  quality  of  working  life  (job  satisfaction,  stress),  (3)  perceived 
quality  and  safety  of  care  provided,  (4)  patient  safety  climate  in  the  outpatient 
surgery  center,  and  (5)  demographics  and  background  infonnation.  Additional 
information  on  the  employee  questionnaire  can  be  found  in  a  paper  to  be 

1 2 

published  in  Advances  in  Patient  Safety:  From  Research  to  Implementation. 

Patient  survey 

The  objective  of  the  patient  survey  was  to  evaluate  (1)  symptom  management; 
(2)  patient  understanding  of  medication,  surgery  preparation  and  post-op 
instructions;  and  (3)  demographics  and  background  information.  We  examined 
the  impact  of  the  intervention  on  symptom  management  and  patient 
understanding.  Additional  infonnation  on  the  patient  survey  can  be  found  in  a 
paper  to  be  published  in  Advances  in  Patient  Safety:  From  Research  to 
Implementation. 13 

Patients  who  presented  a  high  clinical  acuity  for  surgery  or  who  were 
undergoing  high-risk  procedures  were  asked  to  participate  in  the  follow-up 
telephone  interviews.  This  included  patients  classed  as  non-ASA  1,  or  those 
having  procedures  that  have  a  higher  likelihood  of  post-operative  problems.  We 
selected  100  patients  at  one  center,  50  each  at  two  centers,  and  25  patients  at  two 
other  centers  for  each  of  the  data  collection  phases.  First,  patients’  eligibility  for 
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participation  was  determined.  Then,  prior  to  having  surgery,  patients  were 
approached  by  a  nurse  involved  in  their  care  to  detennine  if  they  were  interested 
and  willing  to  participate  in  the  study.  If  the  patient  initially  agreed  to  participate, 
s/he  was  given  an  informed  consent  and  HIPAA  (Health  Insurance  Portability  and 
Accountability  Act  of  1996)  authorization  to  review  and  sign.  Patients  were 
assured  that  no  identifiable  information  concerning  them,  those  providing  care  to 
them,  or  the  surgical  procedure  they  were  undergoing  was  collected  during  the 
interview  and  that  there  would  be  no  “link”  between  the  consent  and  interview 
forms.  Follow-up  telephone  calls  were  placed  to  patients.  If  after  multiple  follow¬ 
up  calls,  the  patient  was  not  reached,  the  patient  was  excluded  from  the  study.  If 
they  were  excluded  their  respective  consent  fonn  was  shredded.  Minors  and  those 
in  “altered  states,”  e.g.,  those  with  dementia  and  known  psychiatric  history,  were 
not  included. 

Data  collection  occurred  over  approximately  four  months,  depending  on  the 
availability  of  the  interviewer  and  patient  to  complete  the  patient  telephone 
interview.  A  63  percent  response  rate  from  patients  who  agreed  to  participate  in 
the  interview  was  attained.  Patients  were  able  to  change  their  mind  at  any  time 
prior  to  or  during  the  phone  interview  and  to  drop  out  of  the  study  without 
penalty. 


Discussion 

The  systems  engineering  intervention  process  described  in  this  paper 
summarizes  a  successful  collaborative  process  between  a  research  team  (Systems 
Engineering  Initiative  for  Patient  Safety  team  comprised  of  engineering  and 
health  care  researchers)  and  five  outpatient  surgery  centers  under  the  auspices  of 
the  Madison  Patient  Safety  Collaborative.  The  success  of  the  collaborative 
process  is  demonstrated  via  continuing  sustained  participation  of  all  surgery 
centers  in  the  research  study.  The  collaborative  process  has  facilitated  data 
collection  by  the  researchers  and  also  has  provided  a  unique  opportunity  to 
demonstrate  the  feasibility  of  implementing  a  systems  engineering  intervention 
aimed  at  improving  both  patient  outcomes  and  individual/organizational 
outcomes.  The  surgery  centers  involved  in  the  systems  engineering  intervention 
process  also  have  largely  benefited  from  interacting  with  the  research  team,  but 
probably  more  importantly  from  interacting  with  each  other.  At  most  of  the 
meetings,  significant  interaction  took  place  directly  between  representatives  of  the 
various  surgery  centers.  They  often  exchanged  information  on  systems  and 
processes  (e.g.,  process  of  patient  follow-up  after  surgery),  on  various  tools  and 
technologies  used  (e.g.,  H&P  forms),  and  procedures  (e.g.,  wrong-site  surgery 
procedure).  Other  city-wide  collaborative  initiatives  led  by  the  Madison  Patient 
Safety  Collaborative  (e.g.,  site  marking  project)  also  have  benefited  from  the 
involvement  of  the  five  surgery  centers  in  our  systems  engineering  intervention 
process.  These  can  be  considered  as  indicators  of  the  success  of  the  SEIPS 
research  study  and  the  intervention  process. 
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Initially  the  bulk  of  the  interaction  between  the  research  team  and  the  surgery 
centers  took  place  in  meetings  involving  all  the  surgery  centers.  In  further 
defining  the  intervention,  the  interaction  with  the  surgery  centers  involved  “one- 
on-one”  activities  with  each  surgery  center.  For  communication  and  coordination 
purposes,  a  member  of  the  research  team  was  designated  as  a  liaison  for  each  of 
the  surgery  centers.  Over  time,  the  interaction  between  the  research  team  and  the 
surgery  centers  evolved  from  a  one-way  interaction  (i.e.,  the  research  team  asking 
the  centers  for  specific  action  or  input)  to  a  two-way  interaction.  The  centers  have 
individually  approached  the  research  team  to  continue  the  project,  which  indicates 
the  value  they  perceive  in  interacting  with  the  SEIPS  research  team. 

Our  systems  engineering  intervention  process  fits  with  the  overall  approach  of 
action  research.14' 15  Action  research  is  a  research  approach  that  focuses  on 
simultaneous  action  and  research  in  a  participative  manner.  The  term  of  action 
research  is  generic  and  refers  to  a  range  of  approaches,  such  as  participatory 
action  research,16  action  science,15  and  reflective  practice.17  Our  SEIPS  research 
study  fits  the  traditional  model  of  action  research  where  a  collaborative  problem¬ 
solving  relationship  was  established  between  a  group  of  researchers  and  five 
outpatient  surgery  centers.  The  double  objective  of  the  SEIPS  study  was  to  (1) 
generate  new  knowledge  (research  objective),  and  (2)  solve  a  problem  (action 
objective).  Center  representatives  repeatedly  commented  that  the  project 
facilitated  and  expedited  change  within  their  respective  organizations  because  the 
“pressure”  associated  with  participating  in  a  collaborative  effort  offered  greater 
impetus  to  change.  In  the  action  research  framework,  the  research  aspect  of  the 
project  was  used  to  produce  pressure  on  each  of  the  five  participating  surgery 
centers  for  implementing  actions  in  response  to  identified  patient  safety  problems. 
The  researchers  provided  data  and  information  that  were  then  used  by  the  center 
representatives  to  foster  and  accelerate  change  in  their  respective  organizations. 
The  researchers  were  often  called  on  by  the  center  representatives  to  present  data 
to  their  staff.  This  seemed  to  increase  the  “credibility”  of  the  data  and  the  change 
proposed  in  response  to  the  problems  identified  by  the  data. 


Conclusion 

The  focus  of  this  paper  is  on  a  detailed  description  of  the  systems  engineering 
intervention  process  based  on  a  clear  conceptual  framework  (i.e.,  the  SEIPS 
model  of  work  system  and  patient  safety)1.  This  detailed  description  can  be  of 
value  to  other  researchers  and  practitioners  interested  in  pursuing  the  avenue  of 
patient  safety  improvement  through  human  factors  and  systems  engineering 
approaches.  The  specifics  of  the  content  of  the  intervention  need  to  be  adapted  to 
the  particular  organization;  however,  the  intervention  process  itself  can  be 
replicated  and  adopted  by  any  organization.  Additional  information  on  the  data 
collection  methods  and  their  results  can  be  found  in  a  series  of  publications  (see 
list  of  references).  Future  publications  will  provide  further  information  on  the 
actual  impact  and  effectiveness  of  the  intervention  with  regard  to  patient 
outcomes  as  well  as  and  individual  and  organizational  outcomes. 
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